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ABSTRACT 


A primary concern in the development of large-scale, real-time, complex, computer-intensive 
systems is ensuring that the system meets the specified requirements. Further, the requirements 
themselves evolve and undergo many changes during the development process. In such a context, 
it is essential to maintain traceability of requirements to various outputs to ensure that the systems 
meets the current set of requirements. 

An empirical study, utilizing focus group and protocol analysis techniques, was conducted 
with students from the Naval Postgraduate School. Their input, along with current literature, was 
used to explore factors to be taken into account while developing a model of traceability, and the 
appropriateness of the two data collection methods in future research. 
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I. INTRODUCTION 


A. OBJECTIVES OF THE RESEARCH 

The goal of this thesis is to conduct an experimental 
study to help derive requirements for designing a traceability 
method at the level of systems design, relating requirements 
to all system components. Such an experiment should provide 
the semantics of the various traceability linkages or 
relationships between requirements and various system 
components. It should also provide mechanisms for reasoning 
with traceability information to support systems development 
and maintenance activities. 

A first step towards accomplishing this objective is to 
understand the critical issues that relate to the capture and 
use of traceability information in systems development. A 
basic premise in the current research, from which results are 
reported in this document, is that development of a model of 
traceability could be geared toward the needs of stakeholders 
at various stages of the systems development process. 

A variety of stakeholders is involved in the systems 
development process, including project sponsors, project 
managers, analysts, designers, maintainers, testing personnel, 
and end users. The approach used in this research to identify 
stakeholders' needs has been empirical. Our study explores 
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the traceability needs of various stakeholders and identifies 
the critical issues that need to be addressed in developing an 
initial model of traceability. This study was conducted with 
graduate students of a systems analysis and design course, in 
a simulated systems development environment. The results of 
this study are being used in designing a comprehensive study 
involving real stakeholders in large-scale, complex, real-time 
systems development efforts. 

Another objective is to evaluate different research tools 
for data collection and analysis to aid in the design of the 
comprehensive study. 

Given the above objectives, the following questions are 
addressed: 

• What are appropriate methodologies that could be used in 
a comprehensive study on traceability in complex, large- 
scale, real-time systems development environments? 

• What are the critical issues in the development of a model 
of traceability supporting various stakeholders in systems 
development? 

B. SCOPE AND LIMITATIONS OF THE STUDY 

The current study employed novice systems designers in a 
simulated design setting. It should be noted that this 
research is designed to provide insights only into issues that 
need to be investigated further, rather than to provide 
conclusive results. Another constraint was the lack of 
resources to evaluate comprehensively the current tools that 
support representation of traceability information. 
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C. METHODOLOGIES 


Two tools were employed in this research: focus group 
interviews for idea generation and protocol analysis to study 
problem-solving behavior. 

D. ORGANIZATION OF DOCUMENT 

Chapter II provides background information on the general 
topic of traceability, a discussion of some of the current 
traceability tools available, and the uses of traceability in 
the Department of Defense (DoD) and Department of the Navy 
(DON) . 

Chapter III describes focus groups and protocol analysis 
and their applications to this research. Each of the 
techniques, and why they were selected for the particular 
requirements of our research, is explained. 

Chapter IV provides an analysis of data collected, 
utilizing focus groups and protocol analysis techniques. It 
discusses the major findings and relates them to current 
literature. This discussion elaborates on the initial 
findings reported by Ramesh and Edwards. (Ramesh and Edwards, 
1992) 

The final chapter presents an initial model of 
traceability, draws conclusions based on research data, and 
makes specific recommendations resulting from the research 
effort. The chapter concludes with recommended areas for 
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additional research. (A more detailed version of 
document is available as Ramesh et al., 1992.) 


- ' 

this 
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II. BACKGROUND 


A. WHAT IS TRACEABILITY? 

A number of different definitions can be provided for 
traceability, depending on the context in which the term is 
used. Norman Schneidewind (1982) depicts traceability as a 
means of maintenance, focusing on the maintenance phase to 
discover sources of error. He defines traceability as "the 
ability to identify the technical information which pertains 
to a software error which has been detected during the 
maintenance phase and thereby trace the error to the 
applicable design specifications and user requirements...." 
(Schneidewind, 1982, p. 4). 

Whereas Schneidewind's concern for traceability is at the 
software level, Greenspan and McGowan (1978) are concerned 
with the use of traceability to effect changes in the entire 
system at various levels. They offer a broader definition of 
traceability as being: 

the property of a system description technique which 
allows changes in one of the three system descriptions— 
requirements, specification, implementation—to be traced 
to the corresponding portions of the other descriptions. 
The correspondence should be maintained throughout the 
lifetime of the system. (Greenspan and McGowan, 1978, p. 
79) 

To achieve the abovementioned correspondence, Agusa et 
al., postulate that two-way traceability is required. They 
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infer traceability is bidirectional by saying, "A requirements 
description is traceable, if each portion of the description 
can be traced to an originating requirement in its 
predecessor, and to a successor description." (Agusa et al., 
1984, p. 226) 

While all of the above definitions focus on 
change/maintenance, other aspects of traceability are not 
emphasized. Michael Edwards offers a more generic and 
inclusive definition of traceability as a technique used to 
"provide a relationship between the requirements, the design, 
and the final implementation of the system." (Edwards, 1991, 
p. 3-8) 

B. TRACEABILITY TOOLS AND CURRENT EXPECTATIONS 

The initial concern with traceability was that of 
providing document traceability. Traceability within 
documents assures that the source of information is 
distinguishable. 

There are a number of existing traceability tools 
developed by industry. Based on the documentation made 
available by software developers, four tools are discussed 
below. Since these tools vary widely in their applications, 
and as yet there are no industry standards for them, no 
attempt at comparison is made. 

One of the earliest systems to capture and use 
traceability data was Automated Requirements Traceability 
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System (ARTS), a bookkeeping program developed to manage the 
requirements of a large, error-prone aerospace system. ARTS 
operates on a data base that includes systems requirements and 
their characteristics. It allows for automated tracking of 
requirements as they are partitioned and apportioned to lower- 
level requirements. ARTS provides upward and downward 
traceability, database management, and output operations on 
requirement-related attributes selected by the user. Like 
ARTS, other current tools often focus on the database 
management issues related to maintaining links between 
requirements and differing components of the system. 

Some of the traceability tools on the market today provide 
for manual parsing and grouping of functional requirements. 
One such tool is Cadre's Tearawork/RQT. Some of this tool's 
other capabilities include point-and-click allocation of 
requirements to targets, navigation through allocation 
channels to integrate the entire life cycle, and the ability 
to propagate allocations between parent and child entities. 

Cadre describes Teamwork/RQT and its concept of 
requirements traceability as follows: 

The purpose of requirements traceability is to reveal 
the mapping between requirements and the deliverable 
components which are intended to satisfy them. Proof of 
compliance is a two-step process; (1) Show the 
correspondence of requirements to deliverable components. 

A table which shows this correspondence is called a 
traceability matrix. (2) Show that the corresponding 
deliverable components correctly satisfy the requirements. 
(Cadre Technologies, Inc., 1990, p. 6) 
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Teledyne Brown Engineering makes the traceability tool 
Requirements Tracer (RT). Concerning its product, Teledyne 
Brown states: 

The RT Tool can be used throughout the entire system 
life cycle (analysis, design, implementation, testing) to 
define, analyze, and trace system requirements. From a 
database of natural language requirements for which 
various criteria have been defined, relationships between 
requirements can be defined. The RT Tool then creates a 
requirements traceability matrix for assistance in 
verifying the proper allocation of all requirements. The 
user may then generate customized reports which output a 
user-selected set of information. (Teledyne Brown 
Engineering, 1991, p. 1) 

Capabilities of RT include such tracing mechanisms as 
parent/child relationships (and how to determine them), 
functional hierarchy, keywords, attributes, querying, 
requirements extraction, and customized report generation. 
Requirements can be allocated to functions or subfunctions by 
either direct entry or selection from a previously defined 
list. 

RTM, made by Marconi Systems Technology, is yet another 
current traceability tool. In discussing traceability 
benefits, the Marconi company affirms; 

In the verification and validation process, traceability 
is the only technique for assessment of consistency 
between different lifecycle phases prior to coding.... 
Ultimately, acceptance testing is a direct assessment of 
the integrated system against the statement of 
requirement, i.e. another form of traceability.... 
Traceability is thus a major technique for risk management 
on a project. (Marconi Systems Technology, 1991, p. 17) 

Requirements and Traceability Management (RTM) provides 
project configurability (specifying where traceability is 
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wanted), requirements engineering, requirements traceability, 
and documentation. More than one type of link is possible 
between objects. 

Current traceability techniques tend merely to provide 
mechanisms to represent relationships without providing any 
guidance on what useful relationships are and how this 
information will be useful during the lifecycle of a system. 
Contemporary methods yield some traceability through simple 
linking techniques that relate requirements to design. These 
methods, however, do not provide any formal definitions of 
traceability linkages. 

C. TRACEABILITY IN DOD AND DON 

As one of the world's major users of large-scale, 
computer-based systems, DoD takes a detailed approach to the 
dilemma of specifying systems requirements. DoD standards may 
provide even an instructive checklist for systems content. 

In February 1988, DoD specified its requirements for 
systems development in its Military Standard DoD-STD-2167A. 
This standard mandates that requirements be traceable through 
the entire system. DoD-STD-2167A formalizes the tracing of 
requirements (in documents) from the original set entailed by 
the customer, to the contractor's written requirements 
specifications, and to the design, test procedures, and 
results. However, the standard states only that traceability 
is required, not what information is to be maintained to 
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achieve this. A clear definition of the types of information, 
or relationships between various systems components that are 
part of a traceability scheme, is lacking. 

Having a precise method for ensuring that requirements are 
met by the design is vital. DoD currently delineates its 
requirements to contractors in documents that are developed by 
numerous specialists in a format that may be thousands of 
pages long. With declining defense dollars, systems must be 
cost-effective, and be able to adapt to major changes during 
their lifecycle. 

One of the foremost issues in developing an efficient and 
effective system involves the maintenance of consistency 
between requirements and design. This consistency entails 
meeting the initial requirements and maintaining requirements, 
design, and implementation consistently throughout the entire 
systems lifecycle. A key element included in a request for 
proposal must be traceability, guaranteeing that the current 
set of requirements is met by the evolving system. 

The current method used by the Navy to specify 
requirements uses mostly a narrative, English format with 
supporting diagrams and charts. Ambiguities are frequent, as 
English specifications are inexact. If specifications are 
formally stated and can be transformed into designs in a 
formal manner, traceability between requirements and designs 
is a by-product of the design process itself. However, since 
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most specifications are written in English, mechanisms are 
needed to capture traceability information explicitly. 

In light of some recent systems malfunctions that produced 
catastrophic consequences (major telephone service shutdowns, 
for example), it is now commonly understood that changes to 
intricate systems can result in unforeseeable and disastrous 
effects to important national defense systems. These problems 
possibly could be avoided if correct traceability methods are 
used along with proper maintenance of systems. 
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III. AN EXPERIMENTAL APPROACH FOR DETERMINING 


TRACEABILITY REQUIREMENTS 


A. INTRODUCTION 

Our data collection strategy involved a two-pronged 
approach: focus group interviews for idea generation and 
evaluation, and protocol analysis of problem-solving behavior. 
This chapter discusses these two techniques and the design of 
the study that employed the two methodologies. Details of the 
research setting and subjects, as well as the reasons for the 
use of data collection techniques, are provided. 


B. FOCUS GROUPS 

1. What Are Focus Groups? 

A focus group interview is a semi-structured exchange 
among a small group of people. It is not a rigidly 
constructed question-and-answer session, nor is it a free 
dialogue between group members; the group has a clear agenda. 
According to Richard Krueger (1988); 

A focus group can be defined as a carefully 
planned discussion designed to obtain perceptions on 
a defined area of interest in a permissive, non¬ 
threatening environment. The discussion is relaxed, 
comfortable, and often enjoyable for participants as 
they share their ideas and perceptions. Group members 
influence each other by responding to ideas and 
comments in the discussion. (Krueger, 1988, p. 18) 
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Focus group interviewing is possibly the most 
consistent qualitative marketing technique in use today. 
(Templeton, 1987, p. 3) Today, in many marketing research 
organizations, group interviews are nearly as common as the 
traditional survey questionnaire. 

Focus groups were originally called focused 
interviews. They were first used in the 1930s by social 
scientists as an alternative to the technique of using an 
interviewer with closed-ended questions and one respondent; 
the idea was that multiple respondents could make comments on 
issues they believed to be important, while interacting with 
one another. In the 1940s, focus groups were used in the 
evaluation of audience responses to radio programs, and the 
observation of the effectiveness of wartime propaganda 
efforts. In the 1990s, although much of what we know about 
the focus group technique has come from market research, all 
professions, from academia to diplomacy and politics, to the 
social science and business worlds, are adopting this 
eminently versatile method. 

The focus group interview is a highly flexible tool 
and as such is extremely popular. Focus groups are 
appropriate for exploratory analysis when little is known 
about a topic; for generating ideas and research hypotheses; 
for determining how groups of individuals think about current 
issues; for producing information, uncovering potential 
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problems, and encouraging creativity. Today, focus group 
interviewing is considered to be a valid scientific method. 
An example of a successful use of this technique is documented 
by Stewart and Sharadasani (1990).‘ 

Focus group interviewing today usually involves seven 
to 12 individuals who discuss a particular topic under the 
direction of a moderator, who promotes discussion and ensures 
that the group stays on the subject. Smaller groups may be 
dominated by one or two members, while larger groups are 
difficult to manage, and limit participation by all members. 
A typical session will last from one-and-one-half to two-and- 
one-half hours. 

2. Effectiveness of Focus Groups 

Focus groups may be used as a method for testing 
hypotheses, especially when the researcher has strong reasons 
to assume his/her hypotheses are correct. Some critics, 
however, maintain that focus groups do not provide "hard" data 


‘The focus group technique was used by the Reagan 
administration in 1988 (an election year) to determine the 
character and extent of the knowledge/opinion gap between the 
American public and government officials, in regard to 
American-Soviet relations. The Reagan team asked two suburban 
Philadelphia focus groups of "average citizens" to examine the 
ways in which a future Soviet-American summit meeting could be 
believably presented to the American people while 
simultaneously garnering popular support. Based on focus 
group responses, the team chose for the trip the theme, "A 
brighter future and a safer world for all people." The 
Philadelphia groups also helped determine some of the events 
of the trip, including with whom President Reagan would meet. 
(Stewart and Shamdasani, 1990, pp. 124-126) 
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and that group members may be atypical of a larger population. 
But even the critics acknowledge that focus groups are useful 
for exploratory research where little is known about a topic. 

The more commonly lauded uses of focus groups 

include: 


• generating research hypotheses that can be submitted to 
further research and testing, using more quantitative 
approaches; 

• stimulating new ideas and creative concepts; 

• diagnosing the potential for problems with a new program, 
service, or product; 

• creating impressions of products, programs, services, 
institutions, or other objects of interest; and 

• learning how respondents talk about the phenomenon of 
interest. 


Some advantages of focus groups include: 


• They are quicker and less costly than individual 
interviews. 

• Direct contact with respondents allows for probing and 
clarification; the respondent can use his own words to 
express himself. 

• Through group interaction, members tend to influence and 
change each others' opinions, and this shift can be 
studied; information and insights are provided that would 
not be available without the group's interaction. 

• Focus groups have a dynamic effect, encouraging 
creativity. 

• Results are believable and easy to understand. 

• There is much research and theory related to focus groups. 
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Some disadvantages of focus groups include: 


• The sample size is limited. 

• Groups may vary widely in their enthusiasm levels and 
responses. 

• Responses are not independent and may be biased by one or 
more participants. 

• Summarization and interpretation of responses may be 
difficult. 

• The moderator has less control in a group setting than in 
a one-on-one interview. 

• The moderator may bias results. 


According to Stewart and Shamdasani: 

We should not overlook the cases in which focus groups 
alone may be a sufficient basis for decision making. One 
example in an applied research setting would be the 
identification of flaws or serious problems with a new 
product or program that would necessitate redesign. 
(Stewart and Shamdasani, 1990, p. 17) 

When little is known about a particular subject, 
there are few good alternatives to focus groups. Focus 
groups are quicker and less expensive than individual 
interviews; one simply must recognize the potential for 
obscuring individual responses. 

3. Group Dynamics 

It is the characteristics of group members in relation 
to one another, and not just individual differentiation, that 
determine group behavior and performance. Focus groups should 
be structured to facilitate the goals of the researcher, while 
avoiding manipulation of the final results. 
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A recurrent supposition regarding focus groups is that 
superior data are obtained when members are strangers. 
However, Stewart and Shamdasani state: 

Generally, focus group sessions are preceded by 'get- 
acquainted' and 'warm-up' sessions that usually provide 
participants ample opportunity to get to know one another. 
Thus, the issue of acquaintanceship appears to be a matter 
of degree in most focus groups, and its influence appears 
modest at best, (ibid, p. 34) 

Another concern regarding focus groups is the members' 
backgrounds. 

In general, interaction is easier when individuals with 
similar socioeconomic backgrounds comprise the group. 
Similarity of abilities, level of intelligence, and 
knowledge tends to facilitate communication at the same 
wavelength. Similarly, in culturally and racially 
homogenous group situations, it may be easier to encourage 
member participation. This suggests that focus groups 
should be designated to maximize interaction by assuring 
similarity with respect to socioeconomic status, (ibid, p. 
38) 


A highly homogenous group may be able to move through 
many questions quickly, while a highly heterogenous group may 
belabor even a couple of questions. But a small degree of 
variation within group characteristics is often a helpful way 
to obtain the contrast and variation that spark lively 
discussions. 

Krueger advises: 

The focus group technique works well when all 
participants are on an equal basis.... Participants should 
be grouped with care. Participants should be placed with 
others at the same level or status in the organization. 
(Krueger, 1988, pp. 96, 167) 
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4 


The Moderator 


When a moderator/interviewer has little experience or 
prior knowledge in a field, the focus group technique can be 
ideal, as David L. Morgan (1988) argued: 

When the researcher is relatively new to an area, or 
puts a priority on not repeating the received wisdom in a 
field, focus groups have much to offer. The fact that 
group interviews can produce useful data with relatively 
little direct input from the researcher may be a distinct 
advantage, especially in comparison to other interviewing 
techniques. (Morgan, 1988, p. 21) 

A designated moderator/interviewer does away with much 
of the distraction associated with the group having to develop 
its own leadership. With respect to the discussion, the 
moderator may be highly directive or very non-directive- 
letting the discussion flow naturally as long as it remains on 
the topic. It is quite common for an interviewer to start 
with some general questions, then focus on more specific 
issues as the group proceeds. 

The amount of direction provided by the interviewer 
does influence the type and quality of the data obtained from 
the group. The amount of structure and direction by the 
moderator must be determined by the broader research agenda, 
including types of information sought, degree of detail the 
information requires, and the manner in which the information 
will be used. 

Discussion of issues relevant to the needs of the 
researcher occur most readily when the moderator takes a more 
directive and structured approach. When this occurs, however. 
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participants discuss what is important to the researcher, not 
necessarily what they consider significant. 

5. The Interview Guide 

In setting the agenda for a focus group, the moderator 
must choose from among research questions to create the 
interview guide. An alternative, available to a researcher 
conducting several focus groups, is the rolling interview 
guide. When multiple groups are involved, the interview guide 
developed for the first group is revised and used for the 
second one, whose guide will again be revised before it can be 
used for the next group, and so on. This technique makes the 
best use of multiple focus groups, permitting information to 
be refined over time as more information is obtained about a 
subject. 

6. Analyzing Focus Group Data 

According to Stewart and Shamdasani; 

The most common purpose of a focus group interview is 
for an in-depth exploration of a topic about which little 
is known. For such exploratory research a simple 
descriptive narrative is quite appropriate. More detailed 
analyses simply are not necessary or efficient. (Stewart 
and Shamdasani, 1990, p. 102) 

For analyzing the content of focus groups, the cut- 
and-paste method is immediate and cost-effective. The use of 
this technique entails reading a transcript and identifying 
sections that are pertinent to the researchers' topic. 
Material related to each topic is then identified and marked. 
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Next, the marked text may be cut out and sorted (using either 
scissors or a computer) for use in the analysis. 

Cut-and-paste is a useful technique, but often relies 
on the judgment of a single analyst. Usually it is 
preferable to have two or more analysts code the focus group 
results independently. 

C. PROTOCOL ANALYSIS 

1. Definition of Protocol Analysis 

Protocol analysis can be defined as "the process of 
translating the chaotic collection of inform-tion, which is 
derived from the protocol, into more useful and meaningful 
representation." (Vitalari and Dickson, 1^583, pp. 948-956) In 
a more general sense, protocol analysis can be thought of as 
the collection and analysis of verbal reports (called 
protocols) made by subjects while they perform a specific 
task. In most cases, protocol analysis is used to generate a 
mechanism for tracing a subject's thought process. Ericsson 
and Simon distinguish between two different types of 
verbalization procedures—retrospective verbalization and 
concurrent verbalization. Retrospective verbalization refers 
to the technique in which the researcher asks the subject for 
information about his/her thought processes after the task is 
completed. Concurrent verbalization, used in this research, 
refers to a technique in which the subject is asked simply to 
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verbalize his/her thought process while working on a task. 
(Ericsson and Simon, 1980, pp. 24-26) 

Concurrent verbalization procedures have been used 
extensively in the study of human problem solving, including 
such areas as general problem-solving behavior, physics 
problem-solving behavior, stock selection, pediatric 
cardiology, and accounting information decisions. (Vitalari, 
1981, p. 82) 

2. Validity of Concurrent Verbalization 

According to Vitalari, despite the extended use of 
concurrent verbalization, considerable contention surrounds 
its use. Some researchers have guestioned the validity of 
verbal reports. The four major issues under contention are: 

• skewed verbalization of true thought process 

• incompleteness 

• interference with thought process 

• subjective bias during analysis 

The first major issue is that if the subject 
articulates his/her own thought process, he/she is allowed to 
decide how it will be verbalized. Therefore, the thought 
process is different from the one verbalized. The second 
issue, incompleteness, argues that the task of verbalizing 
interferes with the main task; hence, the subject is only able 
to verbalize a small part of the actual thought process. The 
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third issue, interference with thought process, refers to the 
researcher probing the subject to explain his/her reasoning, 
etc., during the experiment. The fourth issue, subjective 
bias, occurs if the researchers^ analysis of the data is 
different from what is implied by the verbalizations. 
(Vitalari, 1981, pp. 83-84). However, this bias is prevalent 
in virtually any data analysis technique. 

Some of the ways to safeguard against the above 
problems include ensuring the researchers do not probe the 
subjects during the experiment, and having independent 
researchers analyze the protocols, (ibid, pp. 87, 89) 

3. Evaluating Protocol Analysis Data 

A wide range of methods to evaluate protocol analysis 
data is reported in literature, varying from a quick count of 
the occurrence of certain words in the protocols, to an 
extensive analysis of all the elements in the tasks under 
investigation. The method chosen to analyze the concurrent 
verbalizations in this research was the simple technique of 
searching through the protocols for unique ideas, thoughts, 
etc., relating to traceability issues. 

D. STUDY DESIGN 

1. Subjects and Subject Training Before Experimental 
Study 

A total of 58 subjects were used for this study. They 
came from a Masters program in Information Technology at the 
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Naval Postgraduate School. To prepare the subjects for our 
experiment, they were given a case study conducted in a 
graduate level Systems Analysis and Design course. Each 
subject was a member of a small project team, consisting of 
three or four students each. The case study was based on a 
real-life, large-scale project and had been used successfully 
in similar studies (Ramesh and Dhar, 1992, p. 4), and involved 
processing of customer orders in a utility company. The case 
analysis entailed a variety of data-gathering methods during 
the analysis phase, including informal descriptions of user 
needs, simulated client meetings, and actual documents from 
real-life situations. The major outputs developed by the 
participants included requirements statements, data flow 
diagrams, entity-relationship diagrams, database design, and 
implementation. 

This training case was selected for several reasons: 

• it had been developed after an extensive domain analysis 
was conducted, based on a real-life system developed by a 
large information systems consulting organization; 

• it had been used successfully in several settings, 
including protocol analysis of group problem-solving 
behavior; 

• the problem domain was familiar to the students, as they 
had personal experiences with the services provided by the 
system; 

• real-life data could be easily collected from a utility 
company and used in the analysis and design of the system 
when necessary (e.g., rate schedules were collected from 
the local utility company and used in systems design); 
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• the problem was sufficiently complex to cover all the 
basic elements of systems design; and 

• the problem could be partitioned so that different groups 
of students would be assigned projects that could be 
completed within a reasonable time frame. 

These activities were completed during a period of 
over two months prior to the subjects' participation in the 
focus groups. Many subjects had extensive experience in 
domains other than computer-based systems development, such as 
shipbuilding and aviation maintenance, where concepts of 
traceability are widely recognized. 

2. Task Design Using the Focus Group Technique 

Six focus groups were conducted over a two-week period 
following the subjects' completion of their case studies. 
Each group (approximately eight to 10 subjects) consisted of 
two or three project teams and each session lasted roughly 
one-and-one-half hours. The focus groups were conducted in a 
semiformal setting—a meeting room equipped with facilities for 
audio/video recording. The following steps were utilized for 
each session; 


• a short warm-up period, during which everyone, including 
the moderators, was introduced and the ground rules of the 
interview stated; 

• a predisposition discussion about the traceability issues 
that needed to be explored, including general discussions 
on the various stakeholders' interest in traceability; and 

• a collective and comparative discussion of all topics, 
followed by a wrap-up of the discussion. During this 
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segment, the participants were prompted for their 
summaries of what was discussed in the group meeting. 

3. Task Design Using the Protocol Analysis Technique 


After 

the focus groups 

were conducted, one 

or 

two 

participants 

from 

each project team 

volunteered 

to 

participate 

in the 

protocol 

analysis 

portion 

of 

the 

experiment. 

This 

exercise 

started with each 

subject 


participating in a few short warm-up examples to get him/her 
accustomed to thinking aloud. Following these exercises, 
participants were handed written instructions (see Figure 1), 
followed by a question/answer segment, during which 
clarification of their questions was provided. The exercises 
were conducted individually, with each subject working in a 
semi-private area; his/her thoughts, as they were verbalized, 
were tape recorded. The researchers monitored the sessions to 
operate the audio equipment and to prompt the subjects to 
verbalize their thoughts, when necessary. Each session lasted 
from 30 to 75 minutes. The recordings were transcribed 
verbatim, then searches were conducted throughout the 
transcripts for key words, phrases, concepts, or ideas that 
dealt with issues relating to traceability. Figure 2 
represents a sample transcript from the exercise. 

4 . Rationale for Using Focus Groups and Protocol Analysis 
In light of the information detailed above, we felt 
ourselves to be on firm empirical ground in using focus groups 
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EXERCISB IHSTRUCTIONS 


You hav« just been appointed as the project team member in 
charge of creating and maintaining traceability. You are required 
to review your project and eee if you can find/assign any 
traceability information in the form of relationships or linkages 
between various components during the systems development process 
including: 

(1) requirement s 

(2) Data Flow Diagrams 

(3) ER Models 

(4) Data Base Design 

(5) Implementation 

The traceability information should be geared toward supporting 
various stakeholders involved with the development and use of the 
system. Specifically, you may consider the needs of the following 
categories of stakeholders: 

project sponsor 
project manager 
systems analyst 
systems designer 
maintainer 
end user manager 
end user (other) 
training personnel 

Please attempt to complete the exercise in as much detail as 
possible within the allotted time. It is important that you cover 
every aspect of traceability that you consider important. 
Therefore, you may choose to provide only a few examples of each 
aspect o' traceability information for each category rather than 
being exhaustive. While doing this, also talk about what 
characteristics a tool that might help you in creating and 
maintaining traceability information should have. 

Finally, please note that this exercise is not intended to 
be a test of your problem solving expertise or as an evaluation 
of your project. We are interested in understanding the problem 
solving behavior and are therefore primarily interested in the 
process you go through. Thank you very much for your time. 


Figure l. Exercise Instructions 







Having been exposed to some systems that 
provide relatively good traceability in that the 
traceability can be accomplished to a certain degree 
on line with, with system help, I can tell that we 
don't come anywhere close to that, but that is what 
I would like. I would like for the system to be 
able to answer questions at the various levels. For 
example, for a, for a, at the project sponsor, 
project manager level, I would like them to be able 
to stick in test data which they actually know the 
inputs are and what the outputs should be and 
submit it to the system and ask for the output at 
various levels. For example, after, after they 
collect a certain amount of meter readings, 
eliminating all other factors, and ask how much 
would the charge be and they should be able to, in 
my opinion, get a number ??. And if they do have a 
final bill amount, they should be able to say, OK, I 
want a detailed report of all of the factors that 
went into it-from, from what type of table was used 
to come up with the tax rates, what types of tables 
were used to come up with the rates for a certain 
type of customer such as agricultural, as opposed to 
residential, to how many hours were used at peak 
time. I like the system to be able to answer 
queries to that affect. So, that points, that 
points, if our system were able to do that, that's 
the kind of traceability that, that both our end 
user manager and our end user, end user phone center 
operator, would, would need and find useful. 

(Pause) As far as the ER diagram and data flow 
diagrams, it occurs to me that the place where it 
would, where they, there's two, two place, two 
stakeholders that would, that that would be most 
important to are the system maintainers and the 
systems designers. I know for a fact as we were 
designing the system, we looked, we looked often at 
the requirements kept on going back very, very, very 
often. As a maintainer who does not have the same 
knowledge that we had from the start who has to 
learn the system from scratch after it is built and 
working, he, ER diagrams and data flow diagrams both 
on line and in hard copy as we have them, would 
definitely be important part of traceability. 


Figure 2. Sample Partial Transcript 
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for our research. The following are specific reasons we used 
these techniques: 


• The focus group is a valid, proven research tool in areas 
such as traceability, where not much is known about the 
topic and where generation of ideas and hypotheses for 
further study is desirable. 

• There has been ample research on the focus group technique 
to give us a solid background in using it; at the same 
time, focus groups may be conducted informally enough to 
work well within our academic setting. 

• As mentioned above, when moderators are new to a research 
topic, such as we were to traceability, they are actually 
at an advantage in not reiterating the established 
knowledge of the field. 

• The groups of students attending the Naval Postgraduate 
School were acquaintances who have similar socioeconomic 
backgrounds and levels of intelligence. At the same time, 
there was a small degree of variation (students from 
different Navy backgrounds) that Stuart and Shamdasani 
called for in the groups. 

• Since it is preferable to have two or more analysts coding 
focus group results independently, this technique has 
proven to be suitable for a multi-person research team. 

• The rolling interview technique allowed us to learn as we 
went along and to benefit from conducting multiple groups. 

• As previously mentioned, a simple descriptive narrative, 
rather than technically detailed analysis of the focus 
groups conducted is the most appropriate method for 
analyzing our data. 


In view of the proven track record of protocol 
analysis in numerous diverse areas, it was decided this 
technique could also prove beneficial to our research. Some 
specific reasons for choosing this method include: 


• Protocol analysis has been successfully used previously in 
the area of problem-solving behavior, and conducting an 
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empirical study of the thought processes that various 
stakeholders might have concerning traceability was an 
important focus of this research. 

• A sufficient number of subjects who had prior exposure to 
concepts of traceability in domains other than computer- 
based systems development and who had participated in a 
systems development (as a part of the case study) were 
readily available . 

• The issues under contention, mentioned in Section C above, 
are minimized in our study, since the safeguards discussed 
were implemented during the exercise. 
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IV. RESEARCH FINDINGS AND DISCUSSION 


A. INTRODUCTION 

In this chapter, we discuss results from the analysis of 
data collected during focus groups sessions and protocol 
analysis. First, we review the context in which traceability 
information is likely to be used during systems development, 
i.e., from the perspectives of key stakeholders involved in 
the systems development process. This is followed by a 
discussion of major issues that need to be addressed in the 
development of a ...'■...el of traceability, and the mechanisms to 
support the capture of, and reasoning with, this information. 
Findings from relevant literature are included to elucidate 
the main issues. 

B. STAKEHOLDERS 

A number of stakeholders are involved in the systems 
development process, including project sponsors, project 
managers, analysts, designers, maintainers, and end users. 
The development of a model '■'f traceability should be geared 
towards these various stakeholders in the systems development 
process. This section will address these key stakeholders and 
what their concerns/uses for traceability encompass. 
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1. Project Sponsor 

A project sponsor is the individual or organization 
that provides funding for the system being developed. ("The 
project sponsor is mostly concerned about cost overruns and a 
finished product.")^ Besides assuring the sponsor that 
genuine requirements are met, traceability also provides a 
mechanism to verify that unnecessary "wouldn't it be nice to 
have" features are curtailed. In so doing, the sponsor can 
avoid potential cost overruns, schedule slippages, and similar 
impediments. 

2. Project Manager 

The project manager is the supervisor who "...plans, 
delegates, and controls progress to develop an acceptable 
system within the allotted time and budget." (Whitten, et al, 
1989, p. 99) He/she is the key person held accountable for a 
project from start to finish, and needs to ensure only 
essential requirements are met. In general, he should make 
sure the project is finished on time, within the given budget, 
and that ("the project/system does what it was intended to"). 
A project manager uses such techniques as tracking milestones, 
etc., to ensure his/her responsibilities are accomplished. 
("The project manager needs traceability for...tracking 


'This is a direct quote from a subject participating in 
the protocol analysis exercise. Henceforth, all quotes from 
a subject, made either in a focus group or during the protocol 
analysis exercise, will be enclosed in parentheses and 
quotation marks, but no specific reference will be made. 
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milestones and...keeping tabs on projects.") According to 
Brown, "Traceability provides for ease in determining phase 
completion and product completeness." (Brown, 1987, p. 9) 
Traceability will also help the project manager determine when 
he/she has "covered all requirements so you can stop working 
on a project." (Cadre Technologies, Inc., 1990, p. 4) Based 
on the above reasoning, the project manager is one of the 
primary beneficiaries of traceability. 

3. Systems Designer/Analyst 

"A software designer often needs to trace from 
requirements objects to the corresponding design objects or 
from source code to its corresponding design or requirements 
objects." (Nejmeh, et al, 1989, p. 981) This use of 
traceability will help a systems designer determine if all 
requirements have been considered and specifications 
validated. Further, the designer needs to understand why 
design objects satisfy particular requirements. ("A systems 
designer wants traceability in order to go back to the 
logic.") 

The systems designer is also involved following 
systems implementation. ("To the systems designer, 
traceability is extremely important as far as implementation 
goes . . . [because he] is going to have to accommodate any 
design changes and [determine] the relative impact within the 
organization. If they don't have good traceability in the 
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system, he [systems designer] may implement a change which ... 
may even cripple the system.”) 

4. Maintainer 

Maintainers are the personnel who make repairs to the 
system, once it has been implemented, and updates it to keep 
up with changing requirements. They, in addition to project 
managers mentioned above, are key beneficiaries of 
traceability. Once a change is required, a maintainer needs 
to be able to trace that change back to the requirements that 
necessitated or triggered it, and to pinpoint which parts of 
design/implementation are effected by the change. ("The 
systems maintainer wants traceability for . . . tracing to a 
piece of code, for updating, and for changeability.") 

5. End Users 

Different levels of end users will employ traceability 
in varying degrees. On one end of the spectrum is the casual 
end user who "may only use only a specific on-line program on 
an occasional basis" and "may never become truly comfortable 
with the terminal or the program." (Whitten, et al, 1989, p. 
578) An example of a casual end user is a data entry clerk. 
On the other end of the spectrum is the dedicated end user, 
"one who will spend considerable time using specific on-line 
programs. This user is likely to become comfortable and 
familiar with the terminal's operation." (ibid, p. 578) 
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The casual end user may have little or no need for 
traceability. ("[Casual end users] are pretty much just 
concerned about using the system and don't really care or have 
any power over where it came from or why it is the way it 
is,"; "I wouldn't see where they would be interested in the 
traceability of the design and the functionality of the 
system.") If these end users had access to traceability, it 
could be "dangerous" or could lead to unauthorized changes to 
the system. In particular, ("He [casual end user] is too far 
removed and should not be attempting to change things. He is 
not to be trusted with a 'little' knowledge."; "It is 
dangerous for him to use traceability.") 

Dedicated end users, however, have more applications 
for traceability. Some subjects noted; ("The more 
sophisticated end user needs traceability to manuals to see 
how to achieve the functionality specified in requirements 
documents^ and traceability to programs, via queries, to 
modify them to achieve functionalities."; "For the dedicated 
end user, traceability is beneficial for understanding 
reasoning...and for troubleshooting.") As for the end user 
manager, ("He needs traceability for accountability,^ for 
attempting to improve on a prototype . . . and to enhance 
documentation.") 

^Traceability to documents is addressed in a later 
section. 

^Accountability is addressed in a later section. 
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C. MAJOR ISSUES 


Our studies brought out several issues that need to be 
carefully examined to facilitate the development of a model 
of, and mechanisms to capture and use, traceability 
information. Following are some key issues we discovered, 
both in focus groups and protocol analysis, while evaluating 
the data: 

1. Bidirectional Traceability 

Bidirectional traceability implies both forward and 
backward traceability. Forward traceability is provided if 
each requirement specifically references a design component. 
In other words, forward traceability allows one actually to 
see where requirements materialize in the finished system. In 
the context of software design: 

The forward traceability... is especially important when 
the software product enters the operations and maintenance 
phase. As code and design documents are modified, it is 
essential to be able to ascertain the complete set of 
requirements that may be affected by those modifications. 
(Dorfman and Thayer, 1990, p. 27) 

Backwards traceability is provided when a requirement 
is referenced by a design element. In the context of 
definition of requirements from source documents, "Backward 
traceability...to previous stages of development depends upon 
each requirement explic__iy referencing its source in 
previous documents." (ibid, p. 27) Here, bidirectional 
traceability indicates that a requirement derived from a 
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former requirement has been considered, and that any new 
requirement can be traced back to a preceding one. 


Though one of the most critical uses of traceability 
is ensuring that a design element satisfies a requirement, the 
existence of such a link may not answer the question: are the 
functionalities of the design element required by 
requirements? To help answer this question, links need to be 
bidirectional in order to allow requirements to be traced 
forward from requirements to systems components, and backward, 
from systems components to requirements. 

2. Criticality of Requirements 

A useful way of identifying critical requirements is 
to relate them to the central "mission'' of the system. 
Business processes that generate requirements could be 
identified, and requirements evaluated with respect to such 
processes, to arrive at a classification. For example, 
traceability should address the issue of how the requirements 
are arrived at. This necessitates a mechanism to represent 
the elaboration and refinement of requirements, from the 
central mission or business processes that generate them. 

A good traceability scheme should recognize that all 
requirements are not equal in level of significance or 
criticality. Different levels of detail must be established 
in order to minimize the overhead involved in capturing and 
using traceability information. It may be unnecessary or even 


36 






undesirable, considering the overhead involved in maintaining 
traceability, to maintain linkages between every requirement 
and every output created during the systems design process 
related to it. Costs must be justified by the benefits. It 
is essential to identify critical requirements and maintain 
traceability from those requirements to the various systems 
components. 

The need to relate mission criticality to a 
traceability scheme was considered important by many subjects 
in the focus groups: (”We just have to realize that it 
[traceability] is not necessary for mundane decisions."; 
"Traceability is great for the critical stuff.") 

3. Design Rationale 

Another important component of traceability is design 
rationale information. On the need for design rationale, 
MacLean states: "To understand why a system design is the way 
it is, we also need to understand how it could be different 
and why the choices which were made are appropriate." 
(MacLean, et al, 1989, p. 247) 

Traceability linkages to represent rationale would 
capture the why or reason for design decisions. Design 
rationale allows for reasoning about a system's 
characteristics in the process of understanding and changing 
it. Design rationale is an important issue in change 
management, as it can facilitate change while not necessarily 
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providing the mechanism for doing so. Tracking relationships 
among design objects, and understanding how and which of those 
objects is effected by change, is vital in the maintenance of 
the system. 

The focus group participants were keenly aware of the 
need for design rationale as a component of traceability: 
("The systems designer needs traceability in order to examine 
the logic behind the system.”; “Traceability could be very 
useful for justifying why you did something the way you did 
it."; "Traceability would be good for determining what input 
and output are required."; "We have some artificiality built 
into the system—you can say this is how it's supposed to be, 
but is it really? You may need traceability to help you 
adjust requirements.") 

4 . Project Tracking and Management 

Requirements traceability can be used very 
productively in project management and tracking. During the 
systems definition and subsequent phases, traceability is 
essential to ensure that all systems requirements have been 
met. Establishing all life-cycle phases as complete can go a 
long way toward guaranteeing the ease of the verification and 
validation process. 

The project manager can use links such as status. 
completion date, and authorization between various components 
of the system for scheduling, continuity, and security. Such 
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information is indispensable in integrating project management 
into the systems development operation, and the efficient 
completion of project management tasks. 

Focus group participants were very interested in 
project tracking and management possibilities using 
traceability. ("Without traceability, if you've lost a 
linkage you spend much valuable time tracing back to the 
original requirement."; "If you don't write down your thought 
processes and assumptions, and most people don't, you can't 
remember what you've been doing unless you have 
traceability."; "Humans don't go back to the requirements 
enough."; "Traceability should be extremely helpful with 
tracking costs."; "The project manager needs traceability for 
tracking milestones."/ "Traceability would be great for the 
project manager's security concerns.") 

5. Accountability 

A major use of traceability is to provide 
accountability. Using traceability legitimately to communicate 
with the original designer of a system component, or to 
understand the capability of a system, is an example of such 
potential use. However, caution must be used when employing 
traceability information to enforce accountability. The use of 
accountability information as a means for performance 
appraisal may be inappropriate. A parallel could be drawn to 
the use of information gathered during structured walkthroughs 
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in systems development which should be strictly used for 
understanding and improving the current system and not for 
performance evaluation. 

Some accountability information that could be 
captured, using traceability linkages, include: design 
elements designed by, validated by, and modified by 
development personnel. The availability of such information 
will be indispensable in maintaining and revising a system. 

The focus group subjects perceived an urgent need for 
the accountability element tempered by constraints on its 
usage: ("Traceability needs to be something that humans can 
work with, not just a whip held over people."; "Traceability 
should not be used to threaten people with."; "Accountability 
needs to be supplemented with good communication.") They were 
also mindful of the future: ("Accountability implies 
affordability—we're going to have less resources available in 
the future."; "I'm sure that I'm going to want to look back in 
the future and ask myself who made certain decisions or where 
decisions came from.") 

6. Personnel 

Personnel are a critical component of any large-scale 
system. It just as important to capture how requirements 
relate to personnel as with other components of the system. 
This may involve tracing the responsibility for a requirement 
to a human. 
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A c rehensive mechanism for traceability should link 
the personnel component of a system to the other components. 
Examples of such linkages include systems functionalities 
performed by humans. This information is necessary to ensure 
that the allocation of requirements is complete and correct. 

Focus group participants touched on this concept: 
("We need traceability for human manageability.") 

7. Documents/Manuals 

Document traceability determines the existence of 
relationships between two document segments; it means that a 
particular document is in accord with a previous document, 
with which it has some type of relationship. Document 
traceability also ensures that all components in one document 
have a related component in another document. 

Consistency and completeness constraints apply within a 
document and across documents. Within a requirement 
specification, a requirement description may define inputs 
and outputs which relate to other requirements in the 
specification. Inconsistent references and incomplete 
specifications may occur and can be checked....(Horowitz 
and Williamson, 1986, p. 1079) 


Traceability linkages to documents include interpreted 
by, defined by, and consistent with. Such linkages specify 
how to obtain a required performance from a systems component. 

Our focus group subjects had considerable insight into 
some of the document traceability implications: 
("Stakeholders are interested in having traceability to be 
able to write quality manuals and data dictionaries."; 
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"Traceability is good in that it de-emphasizes [unnecessary 
duplication in] documentation."; "If traceability is good, 
another contractor should be able to do documentation."; 
"Traceability is not a requirement of documentation, but it is 
highly desirable for documentation purposes.") 

8. Dependencies 

Since complex systems are composed of interdependent 
components, such dependencies should be represented and 
maintained. Often the inter-component dependencies are not 
well understood and documented. 

Systems design is a complex activity involving 
interdependent decisions. In the absence of mechanisms to 
record such dependencies, over time and with changing 
development teams, this information will be lost. Such 
dependencies may span different systems components. A 
decision about software may be depend*^ nt on an earlier 
decision about hardware. As the system evolves over its life 
cycle, the hardware decision may be altered, leading to 
inconsistencies with the software that was based on the 
earlier hardware decision. Unless the dependencies are 
captured and maintained, such issues may go undetected, 
leading to severe system integration problems. 

Another form of dependency is the fact that there may 
be several components needed to satisfy a requirement. As the 
system evolves over its development life cycle, it is 
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desirable to identify design or implementation elements that 
"partially satisfy" a given requirement. For instance, a 
hardware/software combination is often necessary to satisfy a 
given requirement. When either the hardware or software 
component is developed, traceability information should 
reflect the fact that the partially satisfied requirements are 
fully satisfied by performance of necessary actions. 

It is possible to identify a combination of design 
elements that satisfy a requirement or are generated by it. 
An example of such a traceability scheme is the use of AND-OR 
graphs to represent traceability linkages. Such AND-OR graphs 
can be used to model a task in terms of a series of goals and 
subgoals. If requirements are treated as goals to be 
satisfied, the successive refinements can be treated as 
subgoals to be satisfied. The goals which can be satisfied 
only when all of their immediate subgoals, are satisfied are 
represented by AND nodes. When goals can be satisfied by any 
of the subgoals, they are represented by OR nodes. Liu and 
Horowitz model the Work Breakdown Structure (WBS) of a 
software project as an AND-OR graph. (Liu and Horowitz, 1989, 
pp. 1282-1283) This concept can be used in maintaining 
traceability linkages between various levels of outputs, when 
a logical combination of lower level outputs satisfies a 
higher level goal or requirement. An AND-OR graph is depicted 
in Figure 3. 
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Figure 3. Example of AND-OR Graph 
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Yet another form of dependency can be summed up: 
("The data base design has a transitive dependency. ") This 
dependency is identified when the ("data base design requires 
the data flow diagram [which in turn] depends on the 
requirements. Therefore, requirements determine database 
design.") 

9. Horizontal and Vertical Traceability 

Vertical traceability refers to the "association of 
software (system) life cycle (SLC) objects of different types 
(typically created in different SLC processes). An example of 
a vertical traceability relationship would be between 
requirements statement and design statement." (Nejmeh, et al, 
1989, p. 981) ("Vertical traceability is easy because there's 
a 'rule'...you explode a process and either you have to or you 
don't.") 

Horizontal traceability refers to the 

association of SLC objects of the same type (typically 
created in the same SLC process) . An example of this type 
of traceability includes parent/child relationships among 
decomposed data flow diagrams and the 'derived from' 
relationship among requirement statements, (ibid, p. 981) 

("When you're moving horizontally, you're analyzing 
what process is inside what process.") Horizontal 
traceability equates to a ("subprocess transferring data to 
another subprocess like primitive levels have to talk to each 
other, etc."). 
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10. Automated Support for Traceability 

Automated support for traceability can be extremely 
v-iluable when systems are large and/or complex. "When 
performed manually, the tasks are time-consuming and error- 
prone; moreover, users' abilities to analyze traceability data 
are limited by the sheer volume of data...." (ibid, p. 981) 
In such circumstances, "an automated software tool is an 
imperative, as the measuring process can become extremely 
onerous." (Shepperd and Ince, 1990, p. 81) As stated by 
Thayer and Dorfman, "There have been many cases where it 
appeared, at the outset, that it would be an easy task to keep 
track of it [manually], but when the system design is 
complete, and the customer is trying to understand whether all 
the test data really satisfies the original requirements they 
wrote, the automated traceability would be 'worth its weight 
in gold'." (Thayer and Dorfman, 1990, p. 66) 

The degree of automated support can vary widely, 
depending on the level of sophistication warranted/desired. 
"The simplest [form] is a list that is tabulated by the ID of 
the requirement." (ibid, p. 66) This list can be changed, as 
needed, to support the iteration process. The use of a 
flexible database program and other more intrica-:e aids can be 
utilized for more complex automated support. 
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D. SUMMARY 


The issues reviewed above suggest there are many aspects 
of traceability which need to be considered when contemplating 
a traceability model for real-time, complex systems. This 
chapter specifically indicated that different stakeholders 
will have different uses for traceability, and in varying 
degrees. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


Based on the findings discussed in Chapter IV, 
recommendations on methodologies to be used in a comprehensive 
study, and the appropriateness of the two techniques used in 
this research, are presented. The need for a model of 
traceability is also addressed. 

A. METHODOLOGIES FOR FUTURE RESEARCH 

The focus of this thesis is to explore experimental 
methods that could help define tracing requirements in systems 
development, based on a laboratory experiment with 58 subjects 
at the Naval Postgraduate School. This study suggests that 
the focus group technique and protocol analysis (to a lesser 
extent, however) could be used successfully to identify 
tracing requirements. 

Comparison of the two primary data-collection strategies 
offers some insights. Focus groups provided surprisingly 
interesting results. In this exploratory data-collection 
method, the researchers' biases do not constrain the 
participants. It should be noted that the moderators of the 
focus groups were non-experts in the traceability field; 
therefore, the potential for bias was minimal. In our study, 
many participants related concepts of requirements 
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traceability to their experiences in shipbuilding and aircraft 
maintenance, which employ similar concepts. Focus groups, 
including participants with real-life systems development 
experience, are likely to provide valuable information, even 
if the participants are not familiar with current traceability 
tools and technigues, provided they have sufficient interest 
in the concept. As the participants are not restricted by the 
researchers' ideas and predispositions, this methodology often 
will provide new perspectives and approaches to the problem 
being explored. 

Protocol analysis, on the other hand, is likely to provide 
specific data on problem-solving behavior. Useful results can 
be obtained if the behavior is studied in a real-life problem¬ 
solving situation. This requirement, however, severely 
constrains the use of protocol analysis in future work for 
several reasons. First, current methods of capturing and 
reasoning with traceability are inadequate to provide us an 
appropriate real-life problem-solving situation. Second, the 
protocol analysis method is quite costly in terms of demands 
on subjects and researchers. Therefore, the use of this 
methodology should be restricted to a small number of subjects 
in a relatively well-defined area of problem solving (e.g., 
traceability for accountability). 
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B. TRACEABILITY MODEL 

Findings from our study suggest that an initial model of 
traceability is needed. One approach to developing such a 
model is to understand the traceability needs of various 
stakeholders in the systems development process. Our study 
attempted to capture these needs through the eyes of the 
stakeholders. We believe our findings will help future 
researchers develop such a traceability model. 
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